﻿ Time Tracks and Time Segments Rethinking the Way to Look at Texts Dan Cristea “Alexandru Ioan Cuza” University of Iași, Faculty of Computer Science Romanian Academy Iași branch, Institute of Computer Science dcristea@info uaic ro Our perception of time • She remembered the words that somemes, at the me of adolescence, her mother told her "You are a person apprehended, right? Tash Aw: Map of the invisible world SYNASC, Timișoara, 26th September 2016 Our perception of time SYNASC, Timișoara, 26th September 2016 Our perception of time • In the kitchen, waing for the water to boil, Margaret wondered how was it that she jumped so naturally to embrace Adam She could not realize what had made her feel so happy to see him again Tash Aw: Map of the invisible world SYNASC, Timișoara, 26th September 2016 Our perception of time SYNASC, Timișoara, 26th September 2016 Goals • A computational representation of temporal phenomena occurring in literary texts, preparing thus investigation tools that would make possible to tackle these phenomena by automatic analysis • There were readers perceive: – changes of the current direction of time, – flashbacks and flashforwards, – temporal ruptures, – perspectives of different narrators on the same story In this talk • Time can be better visualised as threads that interlace and separate, – much like railways do in complex rail yards: me tracks • Propose an XML notation for these representations • Show a few examples that put in evidence different temporal manifestations – commute, rupture, join and split points • Make a preliminary evaluation of the annotations on a small corpus • Freely think further No conclusions! Our work is framed in… Since the ACL workshop on Spaal and Temporal Reasoning (2001), LREC workshop on Annotaon Standards for Temporal Informaon in Natural Language (2002) • Temporal Reasoning (and temporal logic: Allen, etc ) • Annotation conventions/standards to cope with time as mentioned in language and text • Much interest: – information extraction – question answering – textual entailment SYNASC, Timișoara, 26th September 2016 Related work – A notation: TimeML – TIMEX3: mark up explicit temporal expressions (times, dates, durations, etc ) – SIGNAL annotate function words that indicate how temporal objects are to be related to each other (e g on, during, when, if, etc ) – TLINK - Temporal Links: the temporal relationship holding between events or between an event and a time – SLINK - Subordination Link: relations between two events, or an event and a signal – ALINK - Aspectual Link: relationship between an aspectual event and its argument event – EVENT James Pustejovsky et al (2003) TimeML: Robust Specification of Event and Temporal Expressions in Text, AAAI Technical Report SS-03-07 SYNASC, Timișoara, 26th September 2016 Related work – A system (the TARSQI Toolkit) that recognises: • temporal expressions (TimeEx) • events • relations between them Verhagen & Pustejovsky: Temporal Processing with the TARSQI Toolkit, Coling 2008 SYNASC, Timișoara, 26th September 2016 Related work – Narrative Containers: assumed windows provide left and right boundaries for when unanchored events most likely occurred; size parameterized by Document Creation Time (DCT) and the periodicity of the source – Narrative Scope: the timespan between the earliest and latest events described in the document – Narrative Time: the current temporal anchor for events in a document; temporal focus shifts as the reader moves through the document => the manner in which events are linked (related) to the DCT in TimeML Pustejovsky&Stubbs (2011) Increasing Informativeness in Temporal Annotation, Proceedings of the Fih Law Workshop (LAW V), Portland Related work • The model of Narrative Container and its relations wrt to: – Channel: is the document written or spoken? – Production circumstances: how was the document distributed? broadcast, newswire, daily publication – Style: what format was used to present the information? – Presence of a temporal anchor: whether an article contained a Narrative Time in the first sentence of the document Pustejovsky&Stubbs (2011) Increasing Informativeness in Temporal Annotation, Proceedings of the Fih Law Workshop (LAW V), Portland However… • Fiction never mentioned in these studies – most interest attached to process newswires and broadcasted information • Less or no attention given to the link between events and characters involved – and there is more than the mere role of agent in an event: John met Ana when exing the Opera House He said: Amazing voices! SYNASC, Timișoara, 26th September 2016 We are interested in connected sequences of events • In historical studies, dramaturgy and narratology – 1906: story line: t! Amer Physician Mar 86/1: He had to confess his inability to por do the story-line jusce en – 1907: me line – A line used to illustrate or represent Dav s a chronological sequence of events; a chronology; a e chronological sequence , Jam J W ADAMSON Pract of Instr II iv 275: A me line for English you history… can be made with great advantage when the class is k taking its ﬁrst connected course in English history han T Oxford English Dictionary SYNASC, Timișoara, 27th September 2016 We are interested in connected sequences of events • In NLP – TimeLine: a representation of events which are chronologically ordered, in general specific for an entity (a character or participant in an action, a geographical place or region) Chambers and Jurafsky (2008) Unsupervised Learning of Narrative Event Chains In ACL – StoryLine: groups of interacting TimeLines, or mergers of two or more TimeLines where the same characters or entities are taking part in the action A kind of text structure which is not related to the flow of time Laparra, Aldabe and Rigau (2015) From TimeLines to StoryLines: A preliminary proposal for evaluating narratives In ACL-IJCNLP We are interested in connected sequences of events • In NLP – meline (or fabula): a set of event instances ordered in time (anchored to time expressions or relative to each other) and a set of bridging relations between events with explanatory and predictive value (rising action, climax and falling action) – storyline: the timeline made of those events which have the highest connectivity score ⇒ detect storyline in massive streams of news ⇒ not concerned with deciphering the temporal structure of texts Vossen, Caselli and Kontzopoulou (2015) Storylines for tructuring massive streams of news In ACL-IJCNLP s Motivation and goal • Large texts are different then news – news: sequences of events, opinions, commentaries – novels: events are seldom presented in non-sequential order, same events are interpreted from different perspectives • Characters participate in events • Each has her/his own trajectory/destiny/evolution • Although characters’ trajectories intersect in time, one would like to distinguish/separate them SYNASC, Timișoara, 26th September 2016 The me yard model • A Time Yard (TY) is a representation (resembling a diagram) of the characters’ destinies (a too… precious term) in a text, unfolded on the story time axis • A TY is made of Time Tracks (TTs) • A TT is made of Time Segments (TSs) • A book is a sequence of Time Segments SYNASC, Timișoara, 26th September 2016 Time Tracks • A TT displays the story development of one character or a stable group of characters over a period of story time – TTs are deciphered by the reader as having a unity of characters, action and development – One TT can be narrated by one or more relators – TTs are not necessarily characterised by a unity of place SYNASC, Timișoara, 26th September 2016 Border points of TTs • Time tracks can be bordered by: – start points – where they start, – end points – where they finish, – join points – in which two different time tracks unify, – split points – where one time track splits into two separate time tracks SYNASC, Timișoara, 26th September 2016 Time Segments • A TS groups a sequence of events of one character or a stable group of characters over a period of story time, that is uninterruptedly told in a span of text – one TS is necessarily told in a continuous span of text – in a TS, the story time evolves linearly – one TS can be narrated by only one relator SYNASC, Timișoara, 26th September 2016 Border points in TSs • If inner to TTs, TSs can be bordered by: – rupture points – where the story commutes from one time track onto a different one – commute points – in which, although the story goes on, it is said from a different perspective, by a different storyteller; or it moves forth or back on the same TT • If at the extreme of TTs, TSs can also be bordered by: – start or split points – on the left side – join or stop points – on the right side SYNASC, Timișoara, 26th September 2016 Margaret in the kitchen… • TS1:[In the kitchen, waing for the water to boil, Margaret wondered how was it that] • TS2:[she jumped so naturally to embrace Adam ] • TS3:[She could not realize what had made her] • TS4:[feel so happy to see him again ]” SYNASC, Timișoara, 26th September 2016 TTs and TSs in Margaret&Adam’s episode commute points SYNASC, Timișoara, 26th September 2016 Margaret and her mother… • TS1:[She remembered the words] • TS2:[that somemes, at the me of adolescence, her mother told her "You are a person apprehended, right? ] SYNASC, Timișoara, 26th September 2016 TTs and TSs in Margaret&her mother’s episode SYNASC, Timișoara, 26th September 2016 Split points • TS1:[Up to climb into the truck and disappear after the tarpaulin fall down,] • TS2:[he followed them slowly, with his clumsy gait ] SYNASC, Timișoara, 26th September 2016 Join points • TS1:[It was only when she came to the doorway that she realized] • TS2:[someone was there, lying on the front steps He was a boy, a teenager, almost crouched in the womb ] SYNASC, Timișoara, 26th September 2016 Tash Aw’s “Map of the invisible world” bird’s eye time yard SYNASC, Timișoara, 26th September 2016 Reading the TY diagrams • Misses/discontinuities ⇒ horizontal axis in a TY represents the story-time ⇒ not compulsory that a text includes a complete description of an interval ⇒ segments may not fill contiguously a certain TT ⇒ discontinuities: pauses, misses (interrupted lines in the figure above) SYNASC, Timișoara, 26th September 2016 Reading the TY diagrams • Tracking characters’ destinies (developments) => in bold in the figure: Adam’s trajectory => a mere selection operation in the set of TT elements => produce focused summaries SYNASC, Timișoara, 26th September 2016 Not shown in TY diagrams • Ambiguous temporal placements: when the exact order of TSs cannot be determined => more TSs belong to the same TT => but their order is undefined SYNASC, Timișoara, 26th September 2016 Not shown in TY diagrams • Personal interpretations: => different TSs are told by different relators => the same TS is told by different relators An XML notation – in addition to those of TimeML • Elements marking Time Tracks: – LEFT-EP: “START” | ID of a SPLIT end-point – RIGHT-EP: “STOP” | ID a JOIN end-point • Elements marking Time Segments – IN-TT: ID of a TT – TIME: ID of TIME element – TYPE: NARration, REMembers, SUPposition, GENeral knowledge, FICtions – PERspective: a character (entity) An XML notation • Elements marking time constraints – FROM and TO: IDs of TS – REL: BEFORE, IMMEDIATELY-BEFORE, AFTER, IMMEDIATELY-AFTER, SIMULTANEOUSLY, INCLUDES • Elements marking endpoints – TYPE: “JOIN” or “SPLIT” – TT1 and TT2: IDs of the two TTs that either join or split SYNASC, Timișoara, 26th September 2016 The corpus • Few pages from the same novel annotated by a class of master students in CL – interested to evidence inter-annotator agreement • Tash Aw: Map of the invisible world (Romanian version) – currently under annotation (by Andreea Macovei) – to be used in training a TY parser SYNASC, Timișoara, 26th September 2016 çv çvçv Evaluation • Interested to appreciate interannotator agreement on perceiving TTs and TSs • 2 small fragments from MotIW Students taking TSs as new TT TTs are objective, noticeable Students missing some TTs Continuation • On going annotation of the whole MofIW novel • A TY visualiser under construction • A TY annotator that will work hand in hand with the visualiser SYNASC, Timișoara, 26th September 2016 But we, ourselves, live on time yards… • We live our lives sequentially – I lived within my family until the age of 16, then for two years I went in another city to follow a high school, then for 5 years I was a student in Bucharest, and one year before finishing the faculty I was married, then I came to Iași and 4 years later my first dauther was born, and when she was 5 I got divorced, one year later I married again a s o SYNASC, Timișoara, 26th September 2016 Our TYs • My first daughter’s story… • Our lives meet, then split, then meet again… • We meet many people during our lives, each having her/his own trajectory => like railways SYNASC, Timișoara, 26th September 2016 When I say: I lived within my family unl the age of 16… • Actually, each day I went to school, in the holidays I went to my grandparents house in the country side, and I went many mes to cinema… • … and my parents were not near me all the time => looked at it under the lens, a time segment reveals many more (finer) segments => it also depends on the perspecve… SYNASC, Timișoara, 26th September 2016 A time track perspective • Somebody thinks/believes that somebody else has a certain trajectory => courses of events can be seen differently through different perspectives => sometimes TTs of a community of people are grossly seen intertwined, just like making a single destiny: collectivities, people living in a country => although each has her/his own (developing) TT SYNASC, Timișoara, 26th September 2016 A digression • GENLIAS (www genlias nl) – digitisation and trascription of one of the biggest collection of birth, marriage and death certificates in the world: now approx 15 million certificates on 50 million people => 32 million cert • LINKS – conects the names in certificates => traces the trajectory of the population and rebuilds families SYNASC, Timișoara, 26th September 2016 Just imagine… • … that there is a device that records the TT of each one of us => technologically feasible => perhaps not very much wanted… (sentiment of BB oppression, leaks, etc ) • BUT => pages of history can be automatically written, with a proper intelligent summarisation agent => one can leave parts of her/his life to the posterity th SYNASC, Timișoara, 26 September 2016 An application: MyDiary (ForgetMeNote) • An Assisted Leaving project: in the benefit of old persons in danger of Alzheimer disease… – the Agent “follows” the Master from awaking till going to bed – it perceives walkings, drivings, encounters, etc – at the end of the day the agent engages a dialogue with the Master • to clarify ambiguous moments • to exercise Master’s memory SYNASC, Timișoara, 26th September 2016 An application: “Taking the foreign language teacher with me” • An agent able to teach a tourist the language of the country she/he travels in, in a dynamic, interactive manner – aware of the situation the user happens to be in – assists her/him – offers linguistic support, adaptable to the progress the user is making in learning the language • drilling exercises adapted to the situation, • explanations of the reality around her/him by appealing to cultural idiosyncrasies, • real time support in conversations, etc Acknowledgements • Humanitas Publishing House for the Romanian version of Tash Aw’s novel • The Wylie Agency, UK, for the accept to use for reseach the original English version of Tash Aw’s novel • My class of master students in CL, for debates while establishing the annotation conventions and participation in the first annotation experiment • Andreea Macovei for annotating the novel and co- authoring parts of this research • Emi Peia for implementing the annotator and the TY visualiser SYNASC, Timișoara, 26th September 2016 